What this is....¶

Welcome¶

This notebook can serve as essentially the following... an introduction to Python with an interspersal of my own projects or components of. My focus' have primarily been in: Web Development, API Development and Machine Learning algorithms. However, this is a very reader friendly document. I will be adding to it often so please feel free to visit. Also feel free to contact me at jgcblue9558@gmail.com.

Feel for the document¶

I have made an effort to hit the following points with respect to content and structure:

Utility for the complete beginner;
Recurring topics: I believe that repetition is critical and personal experience has taught me that students don't review enough. So topics are repeated every so often for practice
Interspersal with real-world personal projects I created for my own utility. I think its important to introduce utility and seeking of it where we can.

Enjoy!

Python Basics: Variables, Essential Data Structures, Scopes, etc...¶

Python is a versatile and widely-used programming language known for its simplicity and readability. It's used in a variety of domains, from web development to data analysis to artificial intelligence.

In this section, we'll cover the foundational concepts of Python to provide a solid understanding before we dive into Flask.

Variables and Data Types¶

In Python, variables are used to store data values. Unlike other programming languages, Python has no command for declaring a variable; a variable is created the moment you first assign a value to it. Let's explore some basic data types and variable assignments in Python:

# Integer
x = 5
print(x, type(x))

# Float
y = 5.5
print(y, type(y))

# String
name = 'John'
print(name, type(name))

# Boolean
is_active = True
print(is_active, type(is_active))

# List
fruits = ['apple', 'banana', 'cherry']
print(fruits, type(fruits))

# Tuple
coordinates = (4, 5)
print(coordinates, type(coordinates))

# Dictionary
person = {'name': 'Alice', 'age': 30}
print(person, type(person))

Control Structures¶

Control structures allow you to dictate the flow of your program's execution. This includes making decisions with conditional statements and repeating actions with loops. Let's explore some of the fundamental control structures in Python:

# Conditional Statements

age = 18

if age < 18:
    print('You are a minor.')
elif age == 18:
    print('You just became an adult.')
else:
    print('You are an adult.')

# Loops

# For loop
for i in range(5):
    print(i)

# While loop
count = 0
while count < 5:
    print(count)
    count += 1

Lists and List Manipulation¶

Lists are one of the most versatile data structures in Python. They can hold any type of object, including other lists, and can be expanded or reduced as needed. Let's explore the basics of lists and some common operations.

# Creating a list
fruits = ['apple', 'banana', 'cherry', 'date']

# Accessing elements
first_fruit = fruits[0]
last_fruit = fruits[-1]

# Slicing a list
first_two = fruits[:2]
last_two = fruits[-2:]

# Adding an element
fruits.append('elderberry')

# Removing an element
fruits.remove('banana')

# Popping an element from the end
popped_fruit = fruits.pop()

fruits, first_fruit, last_fruit, first_two, last_two, popped_fruit

Tuples and Sets¶

Tuples are similar to lists in that they can hold multiple elements. However, unlike lists, tuples are immutable, meaning their contents cannot be modified after they are created. Sets, on the other hand, are collections of unique elements without any specific order. Let's delve into these data structures.

# Creating a tuple
colors = ('red', 'green', 'blue')

# Accessing elements in a tuple
first_color = colors[0]

# Tuples are immutable, so this will raise an error
# colors[0] = 'yellow'  # Uncommenting this line will cause an error

# Creating a set
unique_numbers = {1, 2, 3, 3, 4, 4, 5}

# Adding an element to a set
unique_numbers.add(6)

# Removing an element from a set
unique_numbers.remove(1)

colors, first_color, unique_numbers

Dictionaries and Mapping¶

Dictionaries in Python are unordered collections of key-value pairs. They provide a way to store data without relying on indexing, allowing for fast lookups based on the key. Let's explore the basics of dictionaries and some common operations.

# Creating a dictionary
student = {
    'name': 'John Doe',
    'age': 25,
    'course': 'Computer Science'
}

# Accessing elements in a dictionary
student_name = student['name']

# Using the get method to access elements
student_course = student.get('course', 'Unknown')

# Adding a new key-value pair
student['grade'] = 'A+'

# Removing a key-value pair
student_age = student.pop('age')

student, student_name, student_course, student_age

Functions and Function Definitions¶

Functions are fundamental to programming in Python. They allow for code modularity, making it easier to manage, maintain, and reuse code. In this section, we'll explore the basics of defining and calling functions, passing arguments, and returning values.

# Defining a simple function
def greet():
    return 'Hello, World!'

# Calling the function
greeting = greet()

# Defining a function with parameters
def add_numbers(a, b):
    return a + b

# Calling the function with arguments
sum_result = add_numbers(5, 3)

greeting, sum_result

Scope and Lifetime of Variables¶

In Python, the scope of a variable refers to the region of the code where the variable can be accessed or modified. The lifetime of a variable refers to the duration for which the variable exists in memory. Let's delve into these concepts and understand the difference between local and global variables.

# Global variable
global_variable = 'I am a global variable'

# Function that accesses the global variable
def access_global():
    return global_variable

# Function that defines a local variable
def local_variable_function():
    local_variable = 'I am a local variable'
    return local_variable

# Accessing the global variable outside the function
global_access = access_global()

# Accessing the local variable
local_access = local_variable_function()

global_access, local_access

Functions¶

Functions are blocks of reusable code that perform a specific task. They allow for code organization and reusability. In Python, you can define a function using the def keyword. Let's explore how to create and use functions in Python:

# Defining a simple function
def greet():
    return 'Hello, World!'

# Calling the function
print(greet())

# Function with parameters
def add(a, b):
    return a + b

print(add(5, 3))

# Function with default parameters
def power(base, exponent=2):
    return base ** exponent

print(power(5))  # Uses default exponent of 2
print(power(5, 3))  # Uses provided exponent

Classes and Objects¶

Python is an object-oriented programming (OOP) language. At the heart of OOP are classes and objects. A class is a blueprint for creating objects, and objects are instances of classes. Classes encapsulate data and the methods to manipulate that data. Let's explore the basics of classes and objects in Python:

# Defining a simple class
class Dog:
    # Constructor method
    def __init__(self, name, breed):
        self.name = name
        self.breed = breed

    # Method to make the dog bark
    def bark(self):
        return 'Woof!'

# Creating an object (instance) of the class
my_dog = Dog(name='Buddy', breed='Golden Retriever')

# Accessing object attributes
print(my_dog.name)  # Buddy
print(my_dog.breed)  # Golden Retriever

# Calling object methods
print(my_dog.bark())  # Woof!

Modules and Packages¶

As your Python programs grow larger and more complex, it's essential to organize your code. One way to do this is by using modules and packages. A module is a single Python file that contains functions, classes, and variables, while a package is a way of organizing related modules into a single directory hierarchy. Let's explore how to use modules and packages in Python:

# Importing a module
import math

# Using a function from the module
print(math.sqrt(16))  # 4.0

# Importing a specific function from a module
from math import sqrt
print(sqrt(25))  # 5.0

# Importing all functions from a module (not recommended due to potential naming conflicts)
from math import *

# Renaming a module during import
import math as m
print(m.factorial(5))  # 120

File Handling¶

Reading from and writing to files is a common task in programming. Python provides built-in functions and methods to work with files. Let's explore the basics of file handling in Python:

# Writing to a file
with open('sample.txt', 'w') as file:
    file.write('Hello, World!')

# Reading from a file
with open('sample.txt', 'r') as file:
    content = file.read()
    print(content)  # Hello, World!

# Appending to a file
with open('sample.txt', 'a') as file:
    file.write('\nAppended Text.')

# Reading the updated content
with open('sample.txt', 'r') as file:
    content = file.read()
    print(content)  # Hello, World!\nAppended Text.

Flask Basics¶

Flask is a lightweight web framework for Python. It's designed to make getting started quick and easy, with the ability to scale up to complex applications. Flask offers suggestions but doesn't enforce any dependencies or project layout. Let's explore the foundational concepts of Flask to get started with web development in Python.

Setting Up Flask¶

To start using Flask, you first need to install it. You can do this using pip, the Python package manager. Once installed, you can create a basic Flask application. Here's how you can set up Flask:

!pip install Flask

# Basic Flask Application
from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

if __name__ == '__main__':
    app.run()

It Goes Down in the OOP (Object Oriented Programming in Python)¶

Python is a versatile language with a rich standard library and many built-in functionalities. Let's delve deeper into some advanced Python topics, starting with a more in-depth look at functions, followed by classes, and then exploring some built-in modules.

Functions (Continued)¶

We previously introduced the basics of functions. Now, let's explore some advanced aspects of functions in Python, including lambda functions, function arguments, and decorators.

# Lambda Functions
# A lambda function is a small anonymous function.
square = lambda x: x ** 2
print(square(5))  # 25

# Function Arguments
# *args allows for any number of positional arguments
# **kwargs allows for any number of keyword arguments
def func(*args, **kwargs):
    print(args)  # Tuple of positional arguments
    print(kwargs)  # Dictionary of keyword arguments

func(1, 2, 3, a=4, b=5)

# Decorators
# A decorator is a function that modifies another function
def my_decorator(func):
    def wrapper():
        print('Something is happening before the function is called.')
        func()
        print('Something is happening after the function is called.')
    return wrapper

@my_decorator
def say_hello():
    print('Hello!')

say_hello()

Classes and Object-Oriented Programming (Continued)¶

We previously introduced the basics of classes in Python. Now, let's explore some advanced aspects of classes and object-oriented programming, including inheritance, polymorphism, encapsulation, and more.

# Inheritance
# Allows a class to inherit attributes and methods from another class
class Animal:
    def __init__(self, species):
        self.species = species

    def make_sound(self):
        return 'Some sound'

class Dog(Animal):
    def make_sound(self):
        return 'Woof!'

dog = Dog('Canine')
print(dog.species)  # Canine
print(dog.make_sound())  # Woof!

# Polymorphism
# Different classes can be treated as instances of the same class through inheritance
class Cat(Animal):
    def make_sound(self):
        return 'Meow!'

def animal_sound(animal):
    return animal.make_sound()

cat = Cat('Feline')
print(animal_sound(cat))  # Meow!
print(animal_sound(dog))  # Woof!

# Encapsulation
# Restricting access to certain parts of an object
class Car:
    def __init__(self):
        self.__engine = 'V8'  # Private attribute

    def get_engine(self):
        return self.__engine

car = Car()
print(car.get_engine())  # V8

Built-in Modules¶

Python comes with a rich standard library that provides many built-in modules, allowing you to perform a wide range of tasks without the need for external libraries. Let's start by exploring the math module, which provides mathematical functions and constants.

import math

# Constants
print('Pi:', math.pi)
print('Euler\'s number:', math.e)

# Power and logarithmic functions
print('Power (2^3):', math.pow(2, 3))
print('Square root of 16:', math.sqrt(16))
print('Natural logarithm of 2:', math.log(2))
print('Base-10 logarithm of 100:', math.log10(100))

# Trigonometric functions
print('Sine of pi/2:', math.sin(math.pi/2))
print('Cosine of pi:', math.cos(math.pi))
print('Tangent of 0:', math.tan(0))

# Rounding functions
print('Ceiling of 4.2:', math.ceil(4.2))
print('Floor of 4.8:', math.floor(4.8))
print('Round 4.5:', round(4.5))
print('Round 4.4:', round(4.4))

Lambda Functions and Comprehensions¶

Now we are going to touch a upon a rather interesting few topics. These have (to the author) a distinctly mathematical feel).

Lambda functions are small, anonymous functions that can have any number of arguments but only one expression. Comprehensions provide a concise way to create lists, dictionaries, and sets based on existing iterables. Let's delve into these topics with examples.

# Basic Lambda Function
multiply = lambda x, y: x * y
print(multiply(5, 3))  # 15

# Using Lambda with filter()
numbers = [1, 2, 3, 4, 5, 6]
even_numbers = list(filter(lambda x: x % 2 == 0, numbers))
print(even_numbers)  # [2, 4, 6]

[2, 4, 6]

# Using Lambda with map()
squared_numbers = list(map(lambda x: x**2, numbers))
print(squared_numbers)  # [1, 4, 9, 16, 25, 36]

[1, 4, 9, 16, 25, 36]

# List Comprehension
squared_list = [x**2 for x in numbers]
print(squared_list)  # [1, 4, 9, 16, 25, 36]

# List Comprehension with Condition
even_squares = [x**2 for x in numbers if x % 2 == 0]
print(even_squares)  # [4, 16, 36]

# Dictionary Comprehension
number_dict = {x: x**2 for x in numbers}
print(number_dict)  # {1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36}

# Set Comprehension
unique_squares = {x**2 for x in numbers}
print(unique_squares)  # {1, 4, 9, 16, 25, 36}

# Nested List Comprehension
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = [num for row in matrix for num in row]
print(flattened)  # [1, 2, 3, 4, 5, 6, 7, 8, 9]

Classes (Advanced Concepts)¶

We've already introduced the basics of classes in Python. Now, let's delve deeper into advanced aspects of classes, including class methods, static methods, property decorators, and more. These paradigms allow handling more of the nuances that arise in real world applications.

# Class Method
class MyClass:
    class_variable = 'I am a class variable'

    @classmethod
    def class_method(cls):
        return cls.class_variable

print(MyClass.class_method())  # I am a class variable

# Static Method
class AnotherClass:
    @staticmethod
    def static_method(value):
        return f'Static method called with value: {value}'

print(AnotherClass.static_method('Hello'))  # Static method called with value: Hello

# Property Decorator
class Circle:
    def __init__(self, radius):
        self._radius = radius

    @property
    def radius(self):
        return self._radius

    @radius.setter
    def radius(self, value):
        if value < 0:
            raise ValueError('Radius cannot be negative')
        self._radius = value

circle = Circle(5)
print(circle.radius)  # 5
circle.radius = 10
print(circle.radius)  # 10

# Inheritance
class Animal:
    def speak(self):
        return 'Animal speaks'

class Dog(Animal):
    def speak(self):
        return 'Dog barks'

dog = Dog()
print(dog.speak())  # Dog barks

# Polymorphism
class Cat(Animal):
    def speak(self):
        return 'Cat meows'

def animal_sound(animal):
    return animal.speak()

cat = Cat()
print(animal_sound(cat))  # Cat meows
print(animal_sound(dog))  # Dog barks

# Encapsulation
class BankAccount:
    def __init__(self):
        self.__balance = 0

    def deposit(self, amount):
        self.__balance += amount
        return self.__balance

    def withdraw(self, amount):
        if amount > self.__balance:
            return 'Insufficient funds'
        self.__balance -= amount
        return self.__balance

account = BankAccount()
print(account.deposit(100))  # 100
print(account.withdraw(50))  # 50

Regular Expressions with the `re` Library¶

I want to take a brief aside to introduce a topic that is vastly underrated with beginners. That is the use of regular expressions. Essentially master over them (they are a topic outside of Python) and related libraries in Python allow the programmer to "search for text" or lines of text in a very powerful way that allows for acting upon documents and using their information in ways that is extremely powerful even essential in many situations.

Regular expressions (regex) are sequences of characters that define a search pattern. The re module in Python provides functions to work with regular expressions. Let's explore some of its functionalities.

# Importing the re module
import re

# Using re.search() to find a pattern
result = re.search('world', 'Hello world!')
print(result.group())  # world

# Using re.findall() to find all occurrences
results = re.findall('\d+', 'There are 123 numbers and 456 more numbers.')
print(results)  # ['123', '456']

# Importing the re module
import re

# Using re.split() to split a string
text = 'apple,banana,grape'
fruits = re.split(',', text)
print(fruits)  # ['apple', 'banana', 'grape']

# Basic Matching using re.search()
text = 'Python is an amazing programming language.'
match = re.search('amazing', text)

if match:
    print('Match found:', match.group())
else:
    print('No match found.')

# Using re.sub() to replace substrings
text = 'I love apples and apples are tasty.'
new_text = re.sub('apples', 'oranges', text)
print(new_text)  # I love oranges and oranges are tasty.

# Using Metacharacters
text = 'The price is $100.'
match = re.search('\$\d+', text)

if match:
    print('Match found:', match.group())
else:
    print('No match found.')

# Using re.compile() to create a regex object
pattern = re.compile('\d+')
results = pattern.findall('There are 123 numbers and 456 more numbers.')
print(results)  # ['123', '456']

# Using re.findall()
text = 'Emails: john.doe@example.com, jane.smith@company.net'
emails = re.findall('[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}', text)
print(emails)

# Using regex groups
text = 'My name is John and I am 30 years old.'
pattern = re.compile('My name is (\w+) and I am (\d+) years old.')
match = pattern.search(text)
name = match.group(1)
age = match.group(2)
print(f'Name: {name}, Age: {age}')  # Name: John, Age: 30

# Using re.sub() for Substitution
text = 'The apple is red and the banana is yellow.'
modified_text = re.sub('apple|banana', 'fruit', text)
print(modified_text)

# Using re.split()
text = 'John,25,Engineer;Jane,30,Doctor'
people = re.split(',|;', text)
print(people)

# Using re.compile() to create a regex object
pattern = re.compile('[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}')
text = 'Contact us at support@example.com'
match = pattern.search(text)

if match:
    print('Match found:', match.group())
else:
    print('No match found.')

Modules and Libraries¶

Modules and libraries are essential components of Python programming. They allow developers to reuse code and leverage pre-existing functionalities, saving time and effort. In this section, we'll explore how to import and use modules and libraries in Python.

# Importing a module
import math

# Using a function from the math module
square_root = math.sqrt(16)

# Importing a specific function from a module
from math import factorial

# Using the imported function
fact_result = factorial(5)

# Importing a module with an alias
import datetime as dt

# Using the module with its alias
current_date = dt.date.today()

square_root, fact_result, current_date

Modules and Libraries¶

Modules in Python provide a way to organize and encapsulate code for better code management and reusability. Libraries, on the other hand, are collections of modules that offer pre-written code to perform common tasks. Let's understand how to import and use modules and libraries in Python.

File Handling and Input/Output¶

File handling allows programs to interact with external files, be it reading data from them or writing data to them. Python provides a built-in open function to open files and various methods to read, write, and manipulate file content. In this section, we'll explore the basics of file handling in Python.

# Importing a module
import math

# Using a function from the math module
square_root = math.sqrt(16)

# Importing a specific function from a module
from math import factorial

# Using the imported function
fact_result = factorial(5)

# Importing a module and giving it an alias
import datetime as dt

# Using the aliased module
current_date = dt.date.today()

square_root, fact_result, current_date

# Writing to a file
with open('sample.txt', 'w') as file:
    file.write('Hello, World!')

# Reading from a file
with open('sample.txt', 'r') as file:
    content = file.read()

# Appending to a file
with open('sample.txt', 'a') as file:
    file.write('\nAppended Text.')

# Reading the updated content
with open('sample.txt', 'r') as file:
    updated_content = file.read()

content, updated_content

File Handling and Input/Output¶

File handling allows us to work with external files, such as reading data from them or writing data to them. Python provides a built-in open function to open files and various methods to read, write, and manipulate file data. Let's explore the basics of file handling in Python.

# Writing to a file
with open('sample.txt', 'w') as file:
    file.write('Hello, World!')

# Reading from a file
with open('sample.txt', 'r') as file:
    content = file.read()

# Appending to a file
with open('sample.txt', 'a') as file:
    file.write('\nAppended Text.')

# Reading the updated content
with open('sample.txt', 'r') as file:
    updated_content = file.read()

content, updated_content

Exception Handling¶

In Python, exceptions can be handled using the try, except, else, and finally blocks. This allows us to catch and respond to specific errors, ensuring that our program can continue running or terminate gracefully. Let's explore the basics of exception handling in Python.

# Demonstrating exception handling
try:
    result = 10 / 0
except ZeroDivisionError:
    result = 'Cannot divide by zero!'
finally:
    final_message = 'Exception handling demonstration completed.'

result, final_message

List Comprehensions¶

List comprehensions are a unique and powerful feature in Python, allowing for the creation of lists in a concise and readable manner. They can be used for various tasks, such as filtering elements, transforming data, or generating new lists based on certain criteria. Let's explore the basics of list comprehensions in Python.

# Using list comprehension to generate a list of squares
squares = [x**2 for x in range(1, 6)]

# Using list comprehension with a condition
even_squares = [x**2 for x in range(1, 6) if x % 2 == 0]

# Using list comprehension with multiple conditions
selected_squares = [x**2 for x in range(1, 6) if x % 2 == 0 and x**2 > 10]

squares, even_squares, selected_squares

Generators and Iterators¶

Generators and iterators are powerful tools in Python for creating and working with sequences of data without the need to store the entire sequence in memory. This can be especially useful for large datasets or for generating infinite sequences. Let's explore the basics of generators, iterators, and the yield keyword in Python.

# Importing the math module
import math

# Using a function from the math module
square_root = math.sqrt(16)

# Importing a specific function from the random module
from random import randint

# Using the imported function
random_number = randint(1, 10)

square_root, random_number

File Handling and Input/Output¶

File handling in Python allows us to work with files on our system, such as reading from or writing to text files, CSV files, or even binary files. Understanding how to handle files is essential for data processing, logging, configuration, and more. Let's explore the basics of file handling.

# Defining a simple generator function
def simple_generator():
    yield 1
    yield 2
    yield 3

# Creating a generator object
gen = simple_generator()

# Using the generator
gen_values = [value for value in gen]

# Defining an iterator
class Counter:
    def __init__(self, limit):
        self.limit = limit

    def __iter__(self):
        self.count = 0
        return self

    def __next__(self):
        if self.count < self.limit:
            self.count += 1
            return self.count
        else:
            raise StopIteration

# Using the iterator
counter = Counter(3)
counter_values = [value for value in counter]

gen_values, counter_values

# Writing to a file
with open('example.txt', 'w') as file:
    file.write('Hello, World!')

# Reading from a file
with open('example.txt', 'r') as file:
    content = file.read()

# Appending to a file
with open('example.txt', 'a') as file:
    file.write('\nAppended Text.')

# Reading the updated content
with open('example.txt', 'r') as file:
    updated_content = file.read()

content, updated_content

Exception Handling¶

Exception handling in Python allows us to deal with unexpected errors that may occur during the execution of our program. By using try-except blocks, we can catch specific errors and handle them in a way that allows our program to continue running or fail gracefully. Let's explore how to use try-except blocks to handle exceptions.

Object-Oriented Programming (OOP) Basics¶

Object-oriented programming (OOP) is a paradigm that organizes data into objects and defines methods to interact with these objects. The main concepts of OOP include classes, objects, inheritance, polymorphism, encapsulation, and abstraction. In this section, we'll introduce the basics of OOP in Python, starting with classes and objects.

# A function that may raise an exception
def divide(a, b):
    return a / b

# Using try-except to handle the exception
try:
    result = divide(10, 0)
except ZeroDivisionError as e:
    result = str(e)

result

# Defining a simple class
class Dog:
    # Class attribute
    species = 'Canis familiaris'

    # Initializer / Constructor
    def __init__(self, name, age):
        self.name = name
        self.age = age

    # Method
    def description(self):
        return f'{self.name} is {self.age} years old'

    # Another method
    def speak(self, sound):
        return f'{self.name} says {sound}'

# Creating objects (instances of the class)
dog1 = Dog('Buddy', 3)
dog2 = Dog('Charlie', 5)

# Accessing object attributes and methods
dog1_description = dog1.description()
dog2_speak = dog2.speak('Woof')

dog1_description, dog2_speak

List Comprehensions¶

List comprehensions are a powerful feature in Python that allow for concise and readable code when generating lists. They can be used to create new lists by applying an expression to each item in an existing list or iterable. Let's explore some examples of list comprehensions.

# Using a for loop to create a list of squares
squares_loop = []
for i in range(10):
    squares_loop.append(i * i)

# Using a list comprehension to create the same list of squares
squares_comprehension = [i * i for i in range(10)]

# Using a list comprehension with a condition
even_squares = [i * i for i in range(10) if i % 2 == 0]

squares_loop, squares_comprehension, even_squares

Inheritance and Polymorphism¶

Inheritance is a fundamental concept in OOP that allows one class to inherit properties and behaviors (methods) from another class. This promotes code reusability and establishes a relationship between the parent (superclass) and child (subclass) classes. Polymorphism, on the other hand, allows us to use a single interface for objects of different types. Let's explore these concepts with examples in Python.

Generators and Iterators¶

Generators and iterators are powerful tools in Python that allow for efficient iteration over large datasets without consuming a lot of memory. Generators generate values on-the-fly, while iterators allow us to traverse these values. Let's explore how to create and use generators and iterators.

# Defining a base (parent) class
class Animal:
    def __init__(self, name):
        self.name = name

    def speak(self):
        return f'{self.name} makes a sound'

# Defining a derived (child) class
class Cat(Animal):
    def speak(self):
        return f'{self.name} says Meow'

class Dog(Animal):
    def speak(self):
        return f'{self.name} says Woof'

# Creating objects
animal = Animal('Generic Animal')
cat = Cat('Whiskers')
dog = Dog('Buddy')

# Demonstrating polymorphism
def animal_speak(animal):
    return animal.speak()

generic_sound = animal_speak(animal)
cat_sound = animal_speak(cat)
dog_sound = animal_speak(dog)

generic_sound, cat_sound, dog_sound

# Creating a simple generator using a function and the yield keyword
def simple_generator():
    yield 1
    yield 2
    yield 3

# Creating an iterator from the generator
gen = simple_generator()

# Using the next function to get values from the generator
first = next(gen)
second = next(gen)
third = next(gen)

# Creating a generator using a generator expression
squared_numbers = (x*x for x in range(5))

list(squared_numbers), first, second, third

Object-Oriented Programming (OOP) Basics¶

Object-oriented programming (OOP) is a paradigm that uses objects and classes for organizing code. It emphasizes encapsulation, inheritance, and polymorphism to create modular and reusable code. In this section, we'll explore the foundational concepts of OOP in Python, starting with classes and objects.

# Defining a simple class
class Dog:
    # A class attribute
    species = 'Canis familiaris'

    # Initializer / Constructor
    def __init__(self, name, age):
        self.name = name
        self.age = age

    # A method
    def description(self):
        return f'{self.name} is {self.age} years old'

    def speak(self, sound):
        return f'{self.name} says {sound}'

# Creating instances of the Dog class
mikey = Dog('Mikey', 6)
rocky = Dog('Rocky', 3)

# Accessing attributes and methods
mikey.description(), mikey.speak('Gruff gruff')

Inheritance and Polymorphism¶

Inheritance allows a class to inherit attributes and methods from another class, promoting code reusability. Polymorphism, on the other hand, enables us to use common interfaces for different data types. Let's explore these concepts with practical examples.

# Defining a base class
class Animal:
    def __init__(self, name):
        self.name = name

    def speak(self):
        raise NotImplementedError('Subclasses must implement this method')

# Defining subclasses
class Dog(Animal):
    def speak(self):
        return f'{self.name} says Woof!'

class Cat(Animal):
    def speak(self):
        return f'{self.name} says Meow!'

# Creating instances of the subclasses
fido = Dog('Fido')
whiskers = Cat('Whiskers')

# Demonstrating polymorphism
fido.speak(), whiskers.speak()

Encapsulation and Abstraction¶

Encapsulation and abstraction are foundational concepts in object-oriented programming. While encapsulation deals with bundling data and methods into a single unit and restricting access to some of the object's components, abstraction focuses on hiding complex implementation details and exposing only the necessary functionalities. Let's explore these concepts with examples.

# Demonstrating Encapsulation

class BankAccount:
    def __init__(self, balance=0):
        self.__balance = balance  # Private attribute

    # Public method to get balance
    def get_balance(self):
        return self.__balance

    # Public method to set balance
    def set_balance(self, amount):
        if amount < 0:
            print('Invalid amount')
            return
        self.__balance = amount

# Creating an instance of BankAccount
account = BankAccount(100)

# Accessing balance using public methods
initial_balance = account.get_balance()
account.set_balance(150)
updated_balance = account.get_balance()

initial_balance, updated_balance

Static and Class Methods¶

In Python, methods within a class can be categorized into instance methods, class methods, and static methods. Each type of method has its own use cases and characteristics. Let's explore these different types of methods with examples.

# Demonstrating Instance, Class, and Static Methods

class MyClass:
    class_variable = 'I am a class variable'

    def __init__(self, value):
        self.instance_variable = value

    # Instance method
    def instance_method(self):
        return f'Instance method called, {self.instance_variable}'

    # Class method
    @classmethod
    def class_method(cls):
        return f'Class method called, {cls.class_variable}'

    # Static method
    @staticmethod
    def static_method():
        return 'Static method called'

# Creating an instance of MyClass
obj = MyClass('I am an instance variable')

# Calling methods
instance_result = obj.instance_method()
class_result = MyClass.class_method()
static_result = MyClass.static_method()

instance_result, class_result, static_result

Decorators and Function Wrappers¶

Decorators are a powerful feature in Python that allow us to wrap another function (or method) to extend or modify its behavior without altering its source code. They are applied using the @ symbol and can be thought of as functions that return another function. Let's dive into some examples to understand decorators better.

# Simple decorator to log function execution

def log_decorator(func):
    def wrapper(*args, **kwargs):
        print(f'Executing function: {func.__name__}')
        return func(*args, **kwargs)
    return wrapper

@log_decorator
def greet(name):
    return f'Hello, {name}!'

# Calling the decorated function
greet('Alice')

Lambda Functions and Anonymous Functions¶

Lambda functions, also known as anonymous functions, are a way to create small, unnamed functions in Python. They are defined using the lambda keyword and can have any number of arguments but only one expression. The expression is evaluated and returned when the lambda function is called. Let's dive into some examples to understand lambda functions better.

# Examples of Lambda Functions

# A simple lambda function to add two numbers
add = lambda x, y: x + y

# Using lambda function with the filter() function
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
even_numbers = list(filter(lambda x: x % 2 == 0, numbers))

# Using lambda function with the map() function
squared_numbers = list(map(lambda x: x**2, numbers))

add(5, 3), even_numbers, squared_numbers

Recursion¶

Recursion is a technique where a function calls itself to solve a problem. Recursive functions break down a problem into smaller instances of the same problem. A crucial aspect of recursive functions is having a base case to prevent infinite recursion. Let's explore recursion with the classic example of calculating the factorial of a number.

# Recursive function to calculate factorial

def factorial(n):
    # Base case
    if n == 0:
        return 1
    # Recursive case
    return n * factorial(n-1)

# Testing the recursive factorial function
factorial_5 = factorial(5)
factorial_7 = factorial(7)

factorial_5, factorial_7

Time and Date Manipulation¶

Handling date and time is a common requirement in programming. Python's datetime module provides a set of tools to work with date and time effectively. Whether you want to represent a specific date, time, or both, or even work with time intervals, the datetime module has you covered. Let's explore some of its functionalities.

import datetime

# Current date
current_date = datetime.date.today()

# Specific date
specific_date = datetime.date(2023, 8, 7)

# Current date and time
current_datetime = datetime.datetime.now()

# Specific time
specific_time = datetime.time(14, 30)  # 2:30 PM

# Time delta (difference between two dates/times)
delta = datetime.timedelta(days=5)
new_date = current_date + delta

current_date, specific_date, current_datetime, specific_time, new_date

Regular Expressions¶

Regular expressions (often abbreviated as regex) are sequences of characters that form a search pattern. They are incredibly powerful for text processing tasks such as searching, matching, and extracting information from strings. Python's re module provides a set of functions to work with regular expressions. Let's explore some basic operations using the re module.

import re

# Sample string
text = 'Email me at john.doe@example.com or at john.doe@mywebsite.org'

# Search for a pattern
match = re.search(r'john.doe@example.com', text)
if match:
    found = match.group()
else:
    found = 'Not found'

# Find all email addresses in the string
email_pattern = r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'
emails = re.findall(email_pattern, text)

found, emails

File Handling and Input/Output¶

File handling allows you to read from and write data to files. Python provides a set of built-in functions and methods for this purpose. Whether you're working with text files, CSVs, or other file formats, Python makes it straightforward to interact with files on your system. Let's explore some basic file operations.

# Writing to a file

# Create or overwrite a file named 'sample.txt' and write some text to it
with open('sample.txt', 'w') as file:
    file.write('Hello, world!\n')
    file.write('Welcome to file handling in Python.\n')

# Reading from a file

# Open the file 'sample.txt' and read its content
with open('sample.txt', 'r') as file:
    content = file.read()

content

Exception Handling¶

Errors and exceptions are inevitable in programming. However, rather than letting the entire program crash when an error occurs, Python provides mechanisms to handle these exceptions gracefully. By using exception handling, you can set up specific responses to different errors, allowing your program to continue running or provide informative error messages to the user. Let's explore how to handle exceptions in Python.

# Basic exception handling

try:
    # This will raise a ZeroDivisionError
    result = 10 / 0
except ZeroDivisionError:
    result = 'Cannot divide by zero!'

# Handling multiple exceptions
try:
    # This will raise a ValueError
    number = int('abc')
except (ValueError, TypeError):
    number = 'Invalid input!'

result, number

List Comprehensions¶

List comprehensions provide a concise way to create lists based on existing lists or other iterables. They can make your code more readable and often more efficient compared to traditional loops. Let's explore some basic and advanced uses of list comprehensions.

# Basic list comprehension
squared_numbers = [x**2 for x in range(10)]

# List comprehension with condition
even_squares = [x**2 for x in range(10) if x % 2 == 0]

# Nested list comprehension
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = [num for row in matrix for num in row]

squared_numbers, even_squares, flattened

Generators and Iterators¶

Generators and iterators allow you to work with large datasets or streams of data without loading everything into memory. They provide a way to iterate over data one item at a time, generating or fetching each item on-the-fly. This can be particularly useful for memory efficiency and performance. Let's explore how to create and use generators and iterators in Python.

# Creating a simple generator using a function with the 'yield' keyword
def simple_generator():
    yield 'Hello'
    yield 'world!'

# Using the generator
gen = simple_generator()
greeting_1 = next(gen)
greeting_2 = next(gen)

# Creating an iterator using a class
class SimpleIterator:
    def __init__(self, limit):
        self.limit = limit
        self.count = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.count < self.limit:
            self.count += 1
            return self.count
        else:
            raise StopIteration

# Using the iterator
it = SimpleIterator(3)
iterator_values = [i for i in it]

greeting_1, greeting_2, iterator_values

Inheritance and Polymorphism¶

Inheritance and polymorphism are core concepts in object-oriented programming that allow for greater modularity and code reuse. Through inheritance, a new class can be derived from an existing class, inheriting its attributes and behaviors. Polymorphism, on the other hand, allows objects of different classes to be treated as objects of a common superclass, enabling more generic and flexible code. Let's explore these concepts in detail.

# Basic Inheritance

class Animal:
    def __init__(self, name):
        self.name = name

    def speak(self):
        return 'Unknown sound'

class Dog(Animal):
    def speak(self):
        return 'Woof!'

class Cat(Animal):
    def speak(self):
        return 'Meow!'

# Creating instances
dog = Dog('Buddy')
cat = Cat('Whiskers')

# Demonstrating polymorphism
animals = [dog, cat]
sounds = [animal.speak() for animal in animals]

sounds

Encapsulation and Abstraction¶

Encapsulation and abstraction are key principles in object-oriented programming that promote modularity, code organization, and data safety. While encapsulation focuses on bundling data and methods into a single unit and controlling access to the data, abstraction is about hiding the complex implementation details and presenting a simplified interface. Let's explore these concepts with practical examples.

# Demonstrating Encapsulation

class BankAccount:
    def __init__(self, balance=0):
        self.__balance = balance  # Private attribute

    # Public method to deposit money
    def deposit(self, amount):
        if amount > 0:
            self.__balance += amount
            return f'Deposited ${amount}. New balance: ${self.__balance}'
        return 'Invalid deposit amount'

    # Public method to withdraw money
    def withdraw(self, amount):
        if 0 < amount <= self.__balance:
            self.__balance -= amount
            return f'Withdrew ${amount}. Remaining balance: ${self.__balance}'
        return 'Invalid or insufficient funds'

    # Public method to check balance (Abstraction in play)
    def check_balance(self):
        return f'Your balance is ${self.__balance}'

# Using the BankAccount class
account = BankAccount()
deposit_result = account.deposit(100)
withdraw_result = account.withdraw(50)
balance = account.check_balance()

deposit_result, withdraw_result, balance

Static and Class Methods¶

Static and class methods are special types of methods in Python that, unlike regular methods, don't operate on instance-specific data. Instead, static methods act as utility functions within the class's namespace, and class methods operate on class-level data. Let's explore these methods with practical examples.

# Demonstrating Static and Class Methods

class Circle:
    pi = 3.14159  # Class attribute

    def __init__(self, radius):
        self.radius = radius

    @staticmethod
    def circle_info():
        return 'A circle is a round shape with no corners.'

    @classmethod
    def unit_circle(cls):
        return cls(1)  # Creates a circle with radius 1

    def area(self):
        return self.pi * self.radius * self.radius

# Using the Circle class
info = Circle.circle_info()  # Calling static method
unit = Circle.unit_circle()  # Calling class method
area = unit.area()

info, area

Decorators and Function Wrappers¶

Decorators in Python offer a powerful and flexible way to modify or enhance functions and methods without altering their actual code. By wrapping functions or methods, decorators can add pre-processing, post-processing, or even completely change the behavior of the target function. Let's dive into the world of decorators with some practical examples.

# Demonstrating Decorators

def simple_decorator(func):
    def wrapper():
        print('Before the function call')
        func()
        print('After the function call')
    return wrapper

@simple_decorator
def say_hello():
    print('Hello!')

say_hello()

Lambda Functions and Anonymous Functions¶

Lambda functions, often referred to as lambda expressions or anonymous functions, are a concise way to create simple functions. They are useful when you need a small function for a short period and don't want to formally define it using the def keyword. The general structure of a lambda function is:

lambda arguments: expression

The expression is evaluated and returned when the lambda function is invoked. Let's see some examples to understand how lambda functions work.

# Examples of Lambda Functions

# A lambda function to add two numbers
add = lambda x, y: x + y
print(add(5, 3))  # Output: 8

# A lambda function to check if a number is even
is_even = lambda x: x % 2 == 0
print(is_even(4))  # Output: True
print(is_even(7))  # Output: False

# A lambda function to get the first character of a string
first_char = lambda s: s[0]
print(first_char('Python'))  # Output: 'P'

Threading and Multi-Threading¶

Threading is a technique that allows a program to run multiple threads concurrently. A thread is the smallest unit of a CPU's execution, and multiple threads can run in parallel, sharing the same memory space. This can lead to more efficient use of system resources, especially in tasks that are I/O-bound or network-bound.

In Python, the threading module provides tools to create and manage threads. Let's explore how to create and work with threads.

import threading
import time

def print_numbers():
    for i in range(1, 6):
        print(i)
        time.sleep(1)

def print_letters():
    for letter in 'ABCDE':
        print(letter)
        time.sleep(1)

# Creating threads
t1 = threading.Thread(target=print_numbers)
t2 = threading.Thread(target=print_letters)

# Starting threads
t1.start()
t2.start()

# Waiting for both threads to complete
t1.join()
t2.join()

print('Both threads have finished execution.')

Threading and Multi-Threading¶

In computing, a thread is the smallest unit of execution that can be scheduled by an operating system. Threads are a way to run multiple operations concurrently in the same process space. Multi-threading refers to the ability of an OS or application to handle multiple threads at once.

Python provides the threading module to work with threads. Using this module, you can create and manage threads to achieve parallel execution of code. However, it's essential to note that due to the Global Interpreter Lock (GIL) in CPython (the standard Python implementation), multiple threads don't execute Python bytecodes in true parallel. This means that while some tasks (like I/O-bound tasks) can benefit from threading, CPU-bound tasks might not see a significant performance improvement.

Let's look at a basic example of creating threads in Python.

import threading
import time

def print_numbers():
    for i in range(1, 6):
        print(f'Number {i}')
        time.sleep(1)

def print_letters():
    for letter in 'abcde':
        print(f'Letter {letter}')
        time.sleep(1)

# Create two threads
thread1 = threading.Thread(target=print_numbers)
thread2 = threading.Thread(target=print_letters)

# Start the threads
thread1.start()
thread2.start()

# Wait for both threads to finish
thread1.join()
thread2.join()

print('Both threads have finished execution.')

Concurrency with the asyncio Library¶

asyncio is a Python library that provides a framework for writing concurrent, asynchronous code using the async and await syntax. It's built around an event loop that can handle multiple I/O-bound tasks without the need for multi-threading or multi-processing.

The primary advantage of asyncio is its ability to handle many tasks concurrently without the overhead of creating and destroying threads. This makes it especially suitable for I/O-bound tasks like network requests, database queries, and file operations.

Let's look at a basic example to understand how asyncio works.

import asyncio

async def say_hello():
    await asyncio.sleep(1)
    print('Hello')

async def say_world():
    await asyncio.sleep(1)
    print('World')

# Create an event loop
loop = asyncio.get_event_loop()

# Run both coroutines concurrently
loop.run_until_complete(asyncio.gather(say_hello(), say_world()))
loop.close()

Context Managers and the `with` Statement¶

Context managers are a convenient way to manage resources in Python, such as files, network connections, and databases. They ensure that resources are properly acquired and released, reducing the chance of resource leaks and making the code cleaner and more readable.

The with statement in Python is used in conjunction with context managers to ensure that setup and teardown actions are taken, typically the acquisition and release of resources.

One of the most common use cases for context managers is file handling. Let's look at an example to understand how the with statement works with files.

# Writing to a file using the 'with' statement
with open('sample.txt', 'w') as file:
    file.write('Hello, World!')

# Reading from the file
with open('sample.txt', 'r') as file:
    content = file.read()
    print(content)

Unit Testing and Test-Driven Development (TDD)¶

Unit testing is a method of testing individual units or components of a software application to determine if they function correctly. A unit can be a function, a method, a class, or any other software component. The primary goal of unit testing is to validate that each unit of the software performs as designed.

Test-Driven Development (TDD) is a software development approach where you write tests before writing the code that will be tested. The process involves three main steps:

Red: Write a failing test.
Green: Write the minimum amount of code to make the test pass.
Refactor: Refactor the code while ensuring the test still passes.

Let's look at a simple example of unit testing in Python using the built-in unittest module.

import unittest

# A simple function to be tested
def add(a, b):
    return a + b

# Test case for the function
class TestAddFunction(unittest.TestCase):
    def test_add(self):
        self.assertEqual(add(1, 2), 3)
        self.assertEqual(add(-1, 1), 0)
        self.assertEqual(add(-1, -1), -2)

# Running the tests
unittest.main(argv=[''], verbosity=2, exit=False)

Debugging Techniques and Tools¶

Debugging is an essential skill for every developer. It involves identifying, isolating, and resolving issues or bugs in a software program. While debugging can sometimes be challenging, various techniques and tools can make the process more manageable.

Here are some common debugging techniques:

Print Statements: One of the simplest and most widely used debugging techniques. By inserting print statements at different points in your code, you can track the flow of execution and inspect variable values.
Automated Tests: Writing tests (like unit tests) can help catch bugs early. When a test fails, it indicates a potential issue in the code.
Code Review: Having another developer review your code can help identify issues that you might have overlooked.
Stepping Through Code: Using a debugger, you can step through your code line by line, inspecting variable values and the flow of execution.
Breakpoints: With a debugger, you can set breakpoints at specific lines in your code. Execution will pause at these points, allowing you to inspect the program's state.

Python provides a built-in debugging tool called pdb (Python Debugger). Let's look at a simple example of how to use pdb to debug a Python program.

import pdb

def buggy_function(x, y):
    pdb.set_trace()  # Setting a breakpoint using pdb
    result = x + y
    result *= 2
    return result

buggy_function(3, '5')  # This will cause a TypeError

Profiling and Performance Optimization¶

Profiling is a technique used to understand the runtime behavior of a program. It provides insights into which parts of the code consume the most time or resources, allowing developers to focus their optimization efforts effectively.

Python provides several tools for profiling, with the most commonly used being the built-in cProfile module. This module offers a way to profile Python programs and get a detailed report on the execution time of individual functions and methods.

Once you've identified performance bottlenecks using profiling, you can then apply various optimization techniques to improve the efficiency of your code.

Let's look at a simple example of how to use cProfile to profile a Python function.

import cProfile

def compute_factorial(n):
    if n == 0:
        return 1
    else:
        return n * compute_factorial(n-1)

# Profile the compute_factorial function
profiler = cProfile.Profile()
profiler.enable()
compute_factorial(20)
profiler.disable()
profiler.print_stats()

Packaging and Distributing Python Modules¶

Once you've developed a Python project, you might want to share it with others or deploy it in different environments. Packaging is the process of bundling your Python project into a distributable format, making it easier to share, install, and deploy.

Python provides tools like setuptools and wheel to help with packaging. The Python Package Index (PyPI) is a repository where you can publish your packages, making them available for others to install using pip.

Here are the general steps to package and distribute a Python project:

Structure Your Project: Organize your project in a directory structure with a clear separation of code, tests, documentation, etc.
Create a setup.py File: This file contains metadata about your project, such as its name, version, dependencies, etc. It's used by setuptools to package your project.
Generate Distribution Packages: Using setuptools, you can generate source distributions (.tar.gz files) and built distributions (.whl files).
Upload to PyPI: Once your packages are ready, you can upload them to PyPI using twine. After uploading, your package will be available for others to install using pip.

Let's look at a basic example of how to create a setup.py file for a hypothetical Python project.

# Example setup.py file for a hypothetical Python project

from setuptools import setup, find_packages

setup(
    name='mypackage',
    version='0.1',
    packages=find_packages(),
    install_requires=[
        'numpy',
        'requests'
    ],
    entry_points={
        'console_scripts': [
            'mycli=mypackage.cli:main',
        ],
    },
    author='Your Name',
    author_email='your.email@example.com',
    description='A simple Python package example',
    license='MIT',
    keywords='example package',
    url='https://github.com/yourusername/mypackage'
)

The above setup.py file provides metadata for a hypothetical Python package named mypackage. Here's a breakdown of the components:

name: The name of the package.
version: The version number of the package.
packages: This uses find_packages() to automatically discover and include all packages in the package directory.
install_requires: A list of dependencies that will be installed with the package.
entry_points: This allows you to specify console scripts that should be available when the package is installed.
author and author_email: Information about the package author.
description: A brief description of the package.
license: The license under which the package is distributed.
keywords: Keywords related to the package.
url: A link to the package's repository or homepage.

Once you have a setup.py file ready, you can use the following commands to package and distribute your project:

python setup.py sdist: This creates a source distribution of your package.
pip install wheel: Install the wheel package, which allows you to create a built distribution.
python setup.py bdist_wheel: This creates a built distribution of your package.
pip install twine: Install the twine package, which allows you to upload your package to PyPI.
twine upload dist/*: This uploads your package to PyPI.

After these steps, your package will be available on PyPI and can be installed by anyone using pip install mypackage.

Next, we'll explore Virtual Environments and Dependency Management.

Virtual Environments and Dependency Management¶

When working on Python projects, it's common to need specific versions of libraries or even Python itself. To avoid conflicts between project dependencies, it's a best practice to use virtual environments. A virtual environment is an isolated environment where you can install packages without affecting the global Python installation or other projects.

Virtual Environments¶

Python includes a module called venv to create virtual environments. Here's how you can create and use a virtual environment:

Creating a Virtual Environment: python -m venv myenv
Activating the Virtual Environment:
- On Windows: myenv\Scripts\activate
- On macOS and Linux: source myenv/bin/activate
Deactivating the Virtual Environment: deactivate

While the virtual environment is active, any packages you install using pip will be installed in the virtual environment rather than the global Python installation.

Dependency Management¶

When you're developing a Python project, it's essential to keep track of the packages and their versions that your project depends on. This is where pip and requirements.txt come into play.

Generating a requirements.txt File: After installing all the packages you need, you can generate a requirements.txt file using the command pip freeze > requirements.txt. This file will list all the packages and their exact versions.
Installing Dependencies from a requirements.txt File: If you have a requirements.txt file, you can install all the listed packages using the command pip install -r requirements.txt.

Using virtual environments and managing dependencies ensures that your project is reproducible and avoids potential conflicts between package versions.

Next, we'll delve into Introduction to Web Development with Python.

Introduction to Web Development with Python¶

Python is a versatile language that can be used for various applications, including web development. Over the years, several frameworks and libraries have been developed to simplify the process of building web applications using Python.

Here are some key concepts and tools associated with web development in Python:

Web Frameworks¶

A web framework is a software framework designed to aid the development of web applications, including web services, web resources, and web APIs. Python has several popular web frameworks, including:

Flask: A lightweight and flexible micro web framework. It's suitable for small to medium-sized applications and offers a lot of freedom in terms of how you structure your application.
Django: A high-level web framework that follows the model-view-controller (MVC) architectural pattern. It's known for its built-in admin interface and ORM (Object-Relational Mapping) system.
FastAPI: A modern, fast web framework for building APIs with Python based on standard Python type hints.

HTTP and REST¶

HTTP (Hypertext Transfer Protocol) is the foundation of any data exchange on the web. When building web applications, you'll often deal with HTTP methods like GET, POST, PUT, DELETE, etc.

REST (Representational State Transfer) is an architectural style for designing networked applications. A RESTful web service uses HTTP requests to perform CRUD (Create, Read, Update, Delete) operations on resources, which are represented as URLs.

Web Templates¶

Web templates allow you to dynamically generate HTML content. Frameworks like Flask and Django come with their templating engines (Jinja2 for Flask and Django's template engine) that allow you to embed Python code within HTML files.

Databases¶

Most web applications need to store data, and this is where databases come into play. Python supports various databases, both SQL (like SQLite, PostgreSQL, MySQL) and NoSQL (like MongoDB). Frameworks like Django come with an ORM that abstracts database operations, making it easier to switch between different databases.

In the upcoming sections, we'll delve deeper into specific topics related to web development with Python, starting with Web Scraping with Beautiful Soup and Requests.

Web Scraping with Beautiful Soup and Requests¶

Web scraping is the process of extracting data from websites. Python offers several libraries to facilitate web scraping, with Beautiful Soup and Requests being among the most popular.

Requests¶

The requests library allows you to send HTTP requests using Python and returns the response as a string. It's a simple yet powerful tool for interacting with web services and retrieving web page content.

Beautiful Soup¶

While requests fetches the web page content, Beautiful Soup is used to parse the content and extract the required data. It provides methods to navigate and search the HTML tree structure, making it easier to extract specific elements from a web page.

Here's a basic example of how to use requests and Beautiful Soup to scrape the title of a webpage:

# Example: Web scraping using requests and Beautiful Soup

import requests
from bs4 import BeautifulSoup

# URL of the webpage to be scraped
url = 'https://www.python.org/'

# Sending a GET request to the webpage
response = requests.get(url)

# Parsing the webpage content with Beautiful Soup
soup = BeautifulSoup(response.content, 'html.parser')

# Extracting the title of the webpage
webpage_title = soup.title.string
webpage_title

# Example: Web scraping using requests and Beautiful Soup

import requests
from bs4 import BeautifulSoup

# URL of the webpage to be scraped
url = 'https://www.wikipedia.org/'

# Sending a GET request to the webpage
response = requests.get(url)

# Parsing the webpage content with Beautiful Soup
soup = BeautifulSoup(response.content, 'html.parser')

# Extracting the title of the webpage
page_title = soup.title.string
page_title

Introduction to Flask (Web Framework)¶

Flask is a lightweight web framework written in Python. It's designed to make getting started with web development quick and easy, with the ability to scale up to complex applications. Flask provides the tools and libraries needed to build web applications, from simple web pages to robust RESTful APIs.

Some of the key features of Flask include:

Microframework: Flask is considered a microframework because it does not require particular tools or libraries. This makes it lightweight and easily customizable.
Integrated Support for Unit Testing: Flask allows for easy unit testing of your applications.
RESTful Request Dispatching: Flask provides a simple way to define and route URLs.
Extensible: Flask can be easily extended with a wide variety of plugins available.

Let's start with a basic example to set up a Flask application:

# Basic Flask application

from flask import Flask

# Create an instance of the Flask class
app = Flask(__name__)

# Define a route and the associated function
@app.route('/')
def hello_world():
    return 'Hello, World!'

# Note: To run the Flask app, you would typically use the following command in the terminal:
# app.run()
# However, we won't execute it here as it will start a server, which isn't supported in this environment.

Templates and HTML Rendering¶

While returning simple strings from Flask routes is useful for basic applications, most web applications require more complex HTML pages. Flask provides a templating engine called Jinja2 that allows for dynamic generation of HTML content.

With Jinja2, you can embed Python-like expressions and control statements within HTML code. This allows for dynamic content generation based on variables and logic defined in your Flask routes.

Here's how you can use Flask with Jinja2 templates:

from flask import render_template

# Sample data
users = [{'name': 'Alice', 'age': 28}, {'name': 'Bob', 'age': 24}, {'name': 'Charlie', 'age': 30}]

@app.route('/users')
def display_users():
    return render_template('users.html', users=users)

# Note: The 'users.html' template would typically be located in a 'templates' folder within your Flask project directory.
# The template might look something like this:
#
# <!DOCTYPE html>
# <html>
# <head>
#     <title>Users</title>
# </head>
# <body>
#     <h2>List of Users</h2>
#     <ul>
#     {% for user in users %}
#         <li>{{ user.name }} - {{ user.age }} years old</li>
#     {% endfor %}
#     </ul>
# </body>
# </html>

Form Handling and User Input¶

Web applications often require user input, which is typically collected through forms. Flask provides tools to handle form submissions and process user input.

When a user submits a form, the data can be sent to the server using either the GET or POST method. The GET method appends form data to the URL in name/value pairs, while the POST method sends the data as part of the request body. For security and privacy reasons, sensitive data like passwords should always be sent using the POST method.

Flask can handle both GET and POST requests and provides easy access to the submitted data.

Let's see how Flask handles form submissions and processes user input:

from flask import Flask, render_template, request

app = Flask(__name__)

@app.route('/login', methods=['GET', 'POST'])
def login():
    if request.method == 'POST':
        username = request.form['username']
        password = request.form['password']
        # Here, you would typically check the username and password against a database or other data store.
        # For simplicity, we'll just check if the username is 'admin' and the password is 'secret'.
        if username == 'admin' and password == 'secret':
            return 'Logged in successfully!'
        else:
            return 'Invalid credentials!'
    return render_template('login.html')

# Note: The 'login.html' template would typically contain a form with fields for the username and password.
# The form's action would be set to '/login' and its method to 'POST'.

Working with Databases in Flask Applications¶

Most web applications require some form of data storage, and relational databases are a common choice. Flask doesn't come with built-in database support, but it's flexible enough to work with various database systems, including SQLite, MySQL, and PostgreSQL.

Flask provides extensions like Flask-SQLAlchemy that simplify database operations. SQLAlchemy is an Object Relational Mapper (ORM) that allows you to interact with your database, like you would with SQL. With an ORM, you create classes that include methods to insert, update, and delete records in the database, without having to write raw SQL statements.

Let's see how we can integrate a database into a Flask application and perform basic CRUD (Create, Read, Update, Delete) operations.

from flask import Flask, render_template, request, redirect, url_for
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///test.db'
db = SQLAlchemy(app)

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(80), unique=True, nullable=False)
    email = db.Column(db.String(120), unique=True, nullable=False)

@app.route('/')
def index():
    users = User.query.all()
    return render_template('index.html', users=users)

@app.route('/add_user', methods=['POST'])
def add_user():
    username = request.form['username']
    email = request.form['email']
    new_user = User(username=username, email=email)
    db.session.add(new_user)
    db.session.commit()
    return redirect(url_for('index'))

# Note: The 'index.html' template would typically display a list of users and a form to add new users.

Creating RESTful APIs with Flask¶

A RESTful API (Representational State Transfer) is an architectural style for designing networked applications. It uses HTTP requests to perform CRUD operations on data. Flask, with its lightweight and modular design, is a great choice for building RESTful APIs.

In a RESTful API, resources are identified by URLs, and standard HTTP methods are used to perform operations on these resources:

GET: Retrieve data
POST: Create data
PUT: Update data
DELETE: Remove data

Let's see how we can create a simple RESTful API using Flask to manage a list of tasks.

from flask import Flask, jsonify, request

app = Flask(__name__)

tasks = [
    {
        'id': 1,
        'title': 'Do the laundry',
        'done': False
    },
    {
        'id': 2,
        'title': 'Write some code',
        'done': True
    }
]

@app.route('/tasks', methods=['GET'])
def get_tasks():
    return jsonify({'tasks': tasks})

@app.route('/tasks/<int:task_id>', methods=['GET'])
def get_task(task_id):
    task = next((task for task in tasks if task['id'] == task_id), None)
    if task is None:
        return jsonify({'error': 'Task not found'}), 404
    return jsonify({'task': task})

@app.route('/tasks', methods=['POST'])
def create_task():
    new_task = {
        'id': len(tasks) + 1,
        'title': request.json['title'],
        'done': False
    }
    tasks.append(new_task)
    return jsonify({'task': new_task}), 201

@app.route('/tasks/<int:task_id>', methods=['PUT'])
def update_task(task_id):
    task = next((task for task in tasks if task['id'] == task_id), None)
    if task is None:
        return jsonify({'error': 'Task not found'}), 404
    task['title'] = request.json.get('title', task['title'])
    task['done'] = request.json.get('done', task['done'])
    return jsonify({'task': task})

@app.route('/tasks/<int:task_id>', methods=['DELETE'])
def delete_task(task_id):
    global tasks
    tasks = [task for task in tasks if task['id'] != task_id]
    return jsonify({'result': True})

In the code above, we've created a simple RESTful API to manage tasks. Here's a brief overview of the routes and their functionalities:

GET /tasks: Returns a list of all tasks.
GET /tasks/<task_id>: Returns details of a specific task based on its ID.
POST /tasks: Creates a new task. The title of the task is expected to be sent as JSON in the request body.
PUT /tasks/<task_id>: Updates an existing task based on its ID. The updated title and/or done status are expected to be sent as JSON in the request body.
DELETE /tasks/<task_id>: Deletes a task based on its ID.

This is a basic example, and in a real-world scenario, you'd likely use a database to store tasks and handle more complex operations and validations.

Consuming APIs using `requests`¶

While Flask allows us to create web applications and APIs, there will be times when you need to interact with external APIs or services. Python's requests library is a popular tool for making HTTP requests and interacting with APIs.

With requests, you can easily send HTTP requests to retrieve or send data to external services. Whether it's a GET request to fetch data or a POST request to send data, requests provides a simple interface to handle these operations.

Let's explore some basic operations using the requests library.

import requests

# Making a GET request
response = requests.get('https://api.github.com')

# Checking the status code of the response
status_code = response.status_code

# Parsing the JSON response
json_response = response.json()

status_code, json_response

Personal Project: API Authentication System Using Flask and the Blueprint Library¶

Now let's get into an application that I set up with the goal of serving a few goals:

Goals

Function as a REST application (Representational State Transfer);
Be flexible enough structurally to allow for the addition of future endpoints;
Function as an authentication system.

With the above requirements in mind I settled upon the following tech stack

Stack

Flask: Extremely flexible, deep assortment of libraries for adding functionality;
Flask-blueprints: this specialized library allows for a more modular approach to the overall functionality of the application, i.e., one can more clearly define folders that handle one endpoint or functionality complete with a deep library of tools I develop;
Flask-JWT, Flask-Praetorian: Libraries that allow for the creation and verification of JavaScript Web tokens, an essential component of the modern day approach toward web authentication.

Web Scraping with Beautiful Soup and Requests¶

Web scraping is the process of extracting data from websites. This can be useful in various scenarios, such as data analysis, data integration, and more. Python offers several libraries for web scraping, and among them, Beautiful Soup is one of the most popular. It provides tools to search, navigate, and modify a parse tree from page source code, making it easier to extract the data you need.

To fetch the web page's source code, we often use the requests library, which we've already discussed. Once we have the page content, we can then use Beautiful Soup to parse and navigate the HTML tree.

Let's see a basic example of how we can use Beautiful Soup and requests to scrape data from a web page.

from bs4 import BeautifulSoup

# Fetching the content of a sample web page
response = requests.get('https://www.example.com')
web_content = response.text

# Parsing the content with Beautiful Soup
soup = BeautifulSoup(web_content, 'html.parser')

# Extracting the title of the web page
page_title = soup.title.string

page_title

What this is....¶

Welcome¶

Feel for the document¶

Python Basics: Variables, Essential Data Structures, Scopes, etc...¶

Variables and Data Types¶

Control Structures¶

Lists and List Manipulation¶

Tuples and Sets¶

Dictionaries and Mapping¶

Functions and Function Definitions¶

Scope and Lifetime of Variables¶

Functions¶

Classes and Objects¶

Modules and Packages¶

File Handling¶

Flask Basics¶

Setting Up Flask¶

It Goes Down in the OOP (Object Oriented Programming in Python)¶

Functions (Continued)¶

Classes and Object-Oriented Programming (Continued)¶

Built-in Modules¶

Lambda Functions and Comprehensions¶

Classes (Advanced Concepts)¶

Regular Expressions with the re Library¶

Modules and Libraries¶

Modules and Libraries¶

File Handling and Input/Output¶

File Handling and Input/Output¶

Exception Handling¶

List Comprehensions¶

Generators and Iterators¶

File Handling and Input/Output¶

Exception Handling¶

Object-Oriented Programming (OOP) Basics¶

List Comprehensions¶

Inheritance and Polymorphism¶

Generators and Iterators¶

Object-Oriented Programming (OOP) Basics¶

Inheritance and Polymorphism¶

Encapsulation and Abstraction¶

Static and Class Methods¶

Decorators and Function Wrappers¶

Lambda Functions and Anonymous Functions¶

Recursion¶

Time and Date Manipulation¶

Regular Expressions¶

File Handling and Input/Output¶

Exception Handling¶

List Comprehensions¶

Generators and Iterators¶

Inheritance and Polymorphism¶

Encapsulation and Abstraction¶

Static and Class Methods¶

Decorators and Function Wrappers¶

Lambda Functions and Anonymous Functions¶

Threading and Multi-Threading¶

Threading and Multi-Threading¶

Concurrency with the asyncio Library¶

Context Managers and the with Statement¶

Unit Testing and Test-Driven Development (TDD)¶

Debugging Techniques and Tools¶

Profiling and Performance Optimization¶

Packaging and Distributing Python Modules¶

Virtual Environments and Dependency Management¶

Virtual Environments¶

Dependency Management¶

Introduction to Web Development with Python¶

Web Frameworks¶

HTTP and REST¶

Web Templates¶

Databases¶

Web Scraping with Beautiful Soup and Requests¶

Requests¶

Beautiful Soup¶

Introduction to Flask (Web Framework)¶

Templates and HTML Rendering¶

Form Handling and User Input¶

Working with Databases in Flask Applications¶

Creating RESTful APIs with Flask¶

Consuming APIs using requests¶

Regular Expressions with the `re` Library¶

Context Managers and the `with` Statement¶

Consuming APIs using `requests`¶